Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New way to edit thresholds #253

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Conversation

MaximMoinat
Copy link
Collaborator

Adds function editThresholds(threshold_file) that can create a thresholds file from a table of modified thresholds.

As described under point 1 in this OHDSI 2021 collaborator showcase: https://www.ohdsi.org/2021-global-symposium-showcase-67/

Making it a draft as it needs to be moved from extras to the main codebase, with Roxygen doc. We do include a readme here. Any feedback at this stage is much appreciated.

@MaximMoinat
Copy link
Collaborator Author

Description from extras/edit_thresholds/Readme.md:

This script can be used to edit selected thresholds in a DQD result. This of course assumes that you have run the DQD, observed it, and have a list of checks for which you would like a different threshold.

About the thresholds

There are 20 different checks. The thresholds for these checks are recorded in one of 3 files: Table, Field or Concept level. This script is valid for editing the three of them simultaneously.
For more information about the checks and their thresholds, visit: https://ohdsi.github.io/DataQualityDashboard/articles/CheckTypeDescriptions.html

About this script

In the previous files, each check can be found using a combination of the check name itself, together with information from additional columns. In summary, the checks and additional columns are the following:

Test Additional columns
TABLE LEVEL
1 measurePersonCompleteness cdmTableName
FIELD LEVEL
2 cdmField [not tested] cdmTableName, cdmFieldName
3 isRequired cdmTableName, cdmFieldName
4 cdmDatatype cdmTableName, cdmFieldName
5 isPrimaryKey cdmTableName, cdmFieldName
6 isForeignKey cdmTableName, cdmFieldName, fkTableName
7 fkDomain cdmTableName, cdmFieldName, fkDomain
8 fkClass cdmTableName, cdmFieldName, fkClass
9 isStandardValidConcept cdmTableName, cdmFieldName
10 measureValueCompleteness cdmTableName, cdmFieldName
11 standardConceptRecordCompleteness cdmTableName, cdmFieldName
12 sourceConceptRecordCompleteness cdmTableName, cdmFieldName
13 sourceValueCompleteness cdmTableName, cdmFieldName
14 plausibleValueLow cdmTableName, cdmFieldName
15 plausibleValueHigh cdmTableName, cdmFieldName
16 plausibleTemporalAfter cdmTableName, cdmFieldName
17 plausibleDuringLife cdmTableName, cdmFieldName
CONCEPT LEVEL
18 plausibleValueLow cdmTableName, cdmFieldName, conceptId, unitConceptId
19 plausibleValueHigh cdmTableName, cdmFieldName, conceptId, unitConceptId
20 plausibleGender cdmTableName, cdmFieldName, conceptId

How to run?

1. Define the checks for which you want to edit the thresholds in a .csv file, including all the 'additional columns', like this:

Level checkName cdmTableName cdmFieldName fkTableName fkDomain conceptId unitConceptId Threshold Notes
Field isRequired MEASUREMENT person_id 10 "one note"
Field plausibleValueLow PERSON year_of_birth 100 "another note"
Field isForeignKey MEASUREMENT person_id PERSON 1
Field fkDomain PERSON race_concept_id Race 49 "and yet another one"
Concept plausibleValueLow MEASUREMENT MEASUREMENT_CONCEPT_ID 2212333 8554 56 "note in another table"

2. Run edit_thresholds.R. Make sure to have the correct file names.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant